Python for finance pt.8 df.corr returns only 1.0 for my entire dataset

by: CAPxSAICIN, 6 years ago


I have my compiled sp500 data in a csv file but when i use df.corr it correlates every piece of data as 1.0, and when I graph the data using the heat map, it shows all of the tickers/data as green ticks meaning that df.corr isnt reading the data for some reason.  my compiled sp500 csv file clearly shows differences in the data but that doesn't seem to transfer over when using the df.corr function, and after searching the internet for hours I cannot find anyone who has experienced this same issue.  here is what I have written so far. (I had to use google finance api so my data set is 'close' instead of adj close):

[def visualize_data():
    df = pd.read_csv('sp500_joined_closes.csv')
    pd.options.display.float_format = '{:.5f}'.format
    #df['AAPL'].plot()
    #plt.show()
    df_corr = df.corr() #creates a correlation table of our data frame.  Generates correlation values      
    print(df_corr.head())
    
    data1 = df_corr.values #gets inner values of our data frame
    fig1 = plt.figure() #specify our figures
    ax1 = fig1.add_subplot(1,1,1) #defined axis 1 by 1 plot 1
    
    heatmap1 = ax1.pcolor(data1, cmap=plt.cm.RdYlGn) #sets the color paramater of heat map (negative,neutral,positive)
    fig1.colorbar(heatmap1)
    ax1.set_xticks(np.arange(data1.shape[0]) + 0.5, minor=False) #sets x ticks for heat map, arranging ticks at every 0.5(half-mark)
    ax1.set_yticks(np.arange(data1.shape[1]) + 0.5, minor=False) #sets y ticks for heat map
    ax1.invert_yaxis() #removes random gap from the top of graph
    ax1.xaxis.tick_top() #moves x axis ticks to the top (meant to look more like a table)
    
    column_labels = df_corr.columns
    row_labels = df_corr.index
    
    ax1.set_xticklabels(column_labels)
    ax1.set_yticklabels(row_labels)
    
    plt.xticks(rotation=90)
    heatmap1.set_clim(-1,1)
    plt.tight_layout()
    #plt.savefig("correlations.png", dpi = (300))
    plt.show()
    
visualize_data()]



You must be logged in to post. Please login or register an account.